Performance and Competence Appraisal in Product Development Teams

Different diagnostic tools are used to evaluate the status and performance of organizations. Typical measurement approaches do not take organizational needs and individual differences into account sufficiently. Advanced measurement of performance is not very typical in product development. The purpose of this paper is to describe the use of two parallel measurement approaches performance measurement and competence analysis. We describe how performance and competence can be measured in organizations, present results from an ongoing case study in a medium-sized Finnish electronics industry company, and present some implications of using performance and competence analysis in parallel. INTRODUCTION Through the intense international competition, demanding markets, and rapidly changing technologies, the interest in knowledge-related work, e.g. engineering and development has increased. For instance in product development, survival in the turbulent market calls for learning to act fast, responding to customer demand, and designing products with distinction and integrity [1]. The efforts to improve product development performance do not always produce expected benefits, however, and it is difficult to determine why. We presume that traditional performance measures should be complemented with softer issues such as measuring competence to produce expected benefits. Below we present our views towards performance measurement and competence appraisal in controlling product development performance and demonstrate how typical methodologies easily neglect actual needs within product development organizations. Performance measurement in product development Motives for performance measurement Performance measures are tools used to follow-up the past performance in an organization or group, predict the level of future performance, and accomplish improvements in performance. Traditional performance measurement systems rely on financial accounting measures, e.g. revenue growth rate, return on investment, market share, and unit costs. Financial measures mainly report on what has happened in the past period without indicating how performance could be improved in the next. For performance improvement purposes, measures indicating learning, growth, performance seen by the customer, and organization’s internal issues should also be used. [2] There are several motives for performance measurement. According to Oman and Pfleeger [3] the three major reasons are understanding, predicting, and controlling. A performance measurement program translates corporate goals and objectives into action. It makes progress towards goals visible, enabling clear communication, objective analysis, and fact-based decision making. Measurement information helps people achieve desired results continuously and identify opportunities for improving results. Measurement can also identify problems or show progress towards goals [4]. The starting point for measuring performance depends on the specific needs of the organization and, therefore, measurement items should vary across different organizations. Typically, performance measures tend to reflect too much the organization chart and, as a result, isolate different functions instead of uniting them into business processes [5]. It is important to measure processes, not organizational functions. Process measures concentrate on the tasks and functions that produce an output [6]. If only organizational functions are measured, there is a danger of sub-optimization. For example, one might limit the measurement to the time used to design new products. Thus, the product testing time and the number of errors detected are neglected. In this case there is a possibility that new products are designed fast but the number of errors detected in the product testing phase, and the time needed to test the products may increase. Measuring the whole process, instead of only parts of it, would prevent this kind of sub-optimization from happening. Content and methods of performance measurement Our approach to performance measurement in product development is to design a balanced set of measures that are derived from the goals of the organization. According to Basili and associates [7], measurement, in order to be effective, must be focused on specific goals, applied to all life-cycle products, processes and resources, and interpreted based on the characteristics and understanding of the organizational context, environment and goals. One framework for developing a balanced set of measures is the Balanced Scorecard approach developed by Kaplan and Norton [8]. In Balanced Scorecard, there are four perspectives from which to choose measures: financial, customer, internal-business-process, and learning and growth perspective. Each of these perspectives is linked to the strategic objectives and competitive demands in the company. Balanced Scorecard provides a balance between external and internal measures. The Scorecard functions as the cornerstone of the current and future success of an organization. Basili’s Goal Question Metric (GQM) is a systematic way of developing measures from goals [9]. The GQM method approach emphasizes that for an organization to measure in a purposeful way it must first specify the goals for itself and its projects, then it must trace those goals to the data that are intended to define those goals operationally, and then provide a framework for interpreting the data with respect to the stated goals. The GQM method ensures that the measures developed fit to the purposes, goals and needs of the organization and are relevant to serving the purpose of performance improvement. When implementing performance measures several practical issues should be taken into consideration. For example, there is the issue of data collection. If the data collection takes too much effort compared to the benefits of the measurement information, it may not worth the effort to implement that particular measure. Secondly, decisions need to be made on who collects the data and how often. Thirdly, design of the performance measurement model should occur in co-operation with all involved. Fourthly, data visualization is one important factor of measurement system implementation, too. With good and simple visualization the measures will be more understandable, and trends are easily detected. Finally, updating of the metrics should be easy, and reporting frequency should also be considered. Competence assessment in product development A number of studies have aimed to identify factors driving superior product development performance, largely from new product development viewpoint but also at product development program and firm level. These factors deal with the strategic intent of the company, product development process, market environment, and organizational issues [10]. The organizational issues section is often largely neglected by designers and engineers. It contains not only the organizational structure and resources but also organizational and team climate and culture, interdepartmental collaboration, job design, employee competencies, leadership, designers’ work motivation, etc. [11]. One starting point for this paper is the assumption that the so called organizational issues are paramount to succeeding in product development and achieving the necessary balance between innovation and creativity, and direction and control [12]. By the side of performance measurement, we particularly focus on job competence and skills that are needed and used in product development work. Competence or skills analysis is often mixed with the term performance appraisal, but it refers rather to a long-term accumulation of knowledge than short term outcomes. Competence management can be considered one part of performance management. The concept of competence In our view, a job competence is an underlying characteristic of an individual that is causally related to criterion-referenced effective and/or superior performance in a job or a situation. [13]. What is relevant here is that competence is context-specific, it is evaluated through some performance criteria, and it is something that the individual is or possesses. Competence is more than skills or knowledge: it includes the person’s motives (or value judgements), traits, and selfconcept (within a social network) [14]. Competence is “the potential to succeed in a situation” including judgements about the context, goals, sufficiency of skills and knowledge, and expected performance. Competence, therefore, is in tight relationship with performance: it is needed to accomplish superior performance repeatedly in a particular context or situation. Competence can be approached at individual, group and organizational level. We view competence at the group level, and would additionally like to make a distinction with core competence which refers to an organizational strategic choice of competitive product and process traits [15]. There are different competence models listing what is relevant to superior performance. For instance Spencer and Spencer [16] have constructed a general competence model consisting of Achievement and action, Helping and human service, Impact and influence, Managerial competencies, Cognitive competencies, and Personal Effectiveness. In each job, most if not all of these competencies are needed, but to varying extents. Some talk about general competencies (shared by all) and specific or individual core competencies (distinguishing between threshold and superior performance in a particular job). What becomes especially important in designing competence studies is the particular job in question, and the organizational strategy and goal-setting of the group under study. Designing competence studies consists of defining performance effectiveness criteria, then identifying the criterion sample (superior and average performers), collecting data, identifying job tasks and competence requirements, validating the model, and determining applications of the model [17]. Assessment of competence As competence is relevant only in context and if matched with performance criteria, the level of competence can be evaluated. We talk of assessing or appraising when referring to evaluating the level or “goodness” of competence. Analysis of competencies can be used e.g. in identifying existing personnel capabilities and gaps between required and existing competencies, estimating the total human capital in an organization, and in determining training needs [18]. It can also be used to identify potential problems in job design and to improve performance. Different mechanisms of assessing competence are presented in Figure 1.